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AflSTRACT 

4 The Comprehensive School Mathematics. Program (CSMP) 

is a program 6t CEMREL, Inc., one of the national educational 
laboratories, and was funded by the National Institute of Education 
(NIE). Its major purpose is the developme'nt of curriculum materials/ 
for kindergarten through grade 6. The original purpose of this draft 
report was to present a comprehensive summary of the mathematics 
achievement of CSMP students in intensive field trials of the 
curriculum. However, as tHe .sponsor has discon,t inued funding, this 
report is likely to be the only one. Therefore, there is a focus on 
the main results of the CSMP evaluation, with many secondary issues 
attended to only in a c^zrsory manner, if at all. Further, there is a 
considerable amount o,f data that relates to issues in mathematics 
education generally which are not presented. 'Readers are encouraged 
to scan the table of contents for an idia of what was originally 
intended, but could not be included. (MP) 
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^ '\ ^ Description of Evaluation ^Report Series 

« 

The Comprehensive School Mathematics Program (CSMP) is a program of * 
CEnIrEL, INC. , pne of the national educational laboratories, ahd is funded by 
the National Irtstitute of Education. Its major purpose is the development of 
curriculum materials for grades K-6. ' ' * 

Beginning ir\ September, 1973, CSMP began an extended 'pi lot trial of its * 
Elementary Program. The pi-lot trial was longitudinal in natur.e; students who 
began uvng CSMP materials in kindergarten or first grade-in 1973-74, were 
able to use them in first and second grades respe'ct^lvely in 1974-75, and so on 
in subsequent y^ars. Hence the adjective "extended". ' 

The evaTuation of the prograoi in thi^s extended pilot trial was iatended 
to be neasohably Vbuifirehensive and to suppiy information desired by a wide 
variety of audiences. For that reason the imports in, this series are ' 
r-ea«onably"non-technical and do not attempt to explor^e •widely some of the 
related issues^. The list of reports for previous years is given on the next 
page. <v ^ ' ^ ^ . - ^ 

) Final Reports in the series are: ^ ^ 

. ^9-A-l Summary of Student Achiev^jnent, Draft Report 
9-A-2 Summary, of Implementation Data; Draft Report 
9-B-l Sixth Grade MANS Test Data 

9-C-l Sixth Grade. Evaluation:. Teacher Questionnaires' 

' The present report, and report 9-A-2, are'sumnary reports describing 

re'sults from -the full nine-ylcir study covering, grades K through 6'. As of this 
time -(November, 1982), these two reports ar$ o'rlly -draft versions and many 
important "issues could not be explored with the careHhat attended'the otherj 
48 -volumes of this series. 



Extended Ptiot Trials of the 
- ^ '* • * Coffip^gehenslve School Mathematics Program 

S» Evaluation Report Series 

Evaluation Report 1-A-l ' Overview. Design i^d Irfetrumentatlon 

'{1974) l-A-Z * External Review of -CSW MateHals«> ^ 

l-A-3- Pmal .Si^ry ^Report . Year 1 

/i-a-l Mid-Year. Jest .Data: CSMP First Gride Content ^ 

• ' • ^ l»ef-2 End-bf-Year Test Data: C3HP First Gr^de Content 

*«l-B-3 Erfd-Kjf-Year Test Oat-a: Standard First Grade Content 

1.^*4 End -of- Year Test Data: CSMP Kindergarten Content 

» Test 04ta on Some General Cognitive Skills 

• l-B-6 Sunmary Test Data: Detroit Schools 

1-C-l Teacher Training Report V 

l-C-a Observations of CSMP First Gride Classes 

* l-C-3 Mid-Year Data from Teacher Questionnaires ^ 
l-C-4 Ehd-of-Year Data from Teacher Questionnaires 
l-C-5 Interviews with CSMP Kindergarten Teachers ^ 

1- C-6 Analysis of Teacher Logs 

Evaluation Report 2-A-l Final Suuinary Report Ye^r 2 

(1975) 2-B-l Second Grade Test O^ta " ^ 

2- B-2 Readmintstration of First Grade Test Items 
2-B-3 Student Interviews 

2-C-l Teacher Questionnaire Data 
2-C-2 Teacher Interviews, Second Grade 
2-C-3 Teacher Interviews, First Grade 

Evaluation Report 3-B-l Second and Third Grade Test Data Year 3 

(1976) 3-C-l Teacher Questionnaire 'Data ^ Year 3 

Evaluation Report 4-A-l Final Sunmary Report Year 4 ^ 

(1977) 4-B-l Standardized Test Data, Third Grade 

4-B-2 Mathematics Applied to Novel Situations (MANS) Test Data 
4-B-3 Individually Administered Problems, Third Grade 

4- C-l Teacher, Questionnaire' Data, Third Grade - 

Evaluation Report 5-B-l Fourth Grade MANS Test Data 

(1978) 5-B-2 Individually Administered Problems, Fourth Grade 

5- C-l Teacher Questionnaire and Interview Data, Fourth Grade ' 

Evaluation Report 6-B-l Cotrparative Test Data: Fourth Gride 

(1979) 6-B-2 Preliminary Test Data: Fifth Grade 
6*C-1 Teacher Questionnaire Data: Grades 3*5 

Evaluation Report 7-B-l - Fifth Grade EvaluatioS: Volume I» Summary 

(1980) 7-B-2 Fifth Grade Evaluation: Volume II, Test Data 
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7-B-4 Re-evaluation of Second Grade, Revised MANS Tests 
7-S-5 , Achievement of Former CSMP^ students at Fourth Grade 

7- B-6 ' Student Achievement, Rapid Implementation Model 

Evaluation Report 8-B-l Sixth Grade Evaluation, Preliminary Study 

(1981) 8-B-2 Evaluation of Revised Second Grade, MANS Slue Level 

8- B-3 * Evaluation of Revised Thifd Grade, MANS Green Level 
\ 8-B-4 ' Three Evaluations of Gifted Student Use 

' * 8-C-l Preliminary Study of CSMP "Graduates" 

Key to Indexing 

Evaluation Reports are labelled m^X-n,- 

where "m\is the year of the pilot study, with 1973-74 as Year 1. 
"X" is the type of data beino reported where A j^^or over- 
views and summaries, B is for student outcomes and C 
'is for other data. , , ' 
"n" is the number within a given year and type of datar 
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I. INTRODUCTION' 
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Since 1973, th€ Comprehensive School Mathematics* Program (€SMP) has been 
developed, pilot^tes.ted and widely used in schools. Adopted by 134 sites 
across the country, it has been a massive^curriculum development Bnd 
implementation ^effort involving over $10 million in federal fuTids. CSMP has 
been the mathematics program for students in 9,000 classrooms throughout the 

country and more than '6,000 teachers have been trained to teach CSMP. To 

V 

assess the program's impact, an independent evaluation has been .conducted, 
including the assessment of tHe mathematics achieverpent of 14,000 students 
nationwide". . ^ 

The original purpos^e of this report was to^present a comprehensive summary 

of the mathematics achievement of CSMP students in these intensive field 

< 

trials of the curriculum. This^ draft report was to be the first, stage in this 
reporting process. Unfortunately, because. of a decisionvby the sponsor to 
discontinue f upding,., the present report is likely to be the only report. ' * • 
Hence, there is a focus on the main results of the CSMP Evaluation and many 
secondary i.ssues must be attended to in only a cursory 'manner, if at all. ^ v 

There is also a considerable amoun^f data that relates to issues in 
mathematics education generally (not just CSMP), which. cannot be presented in' 
this abbreviated report. The interested reader can scan the Table of Contents 
{ for an id^a of what was originally intended for this report. 
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Description of the CSMP Curriculum arrd Its Development Cycle ^JUIi/'^Lij iJ 



CSMP began at Carbondale, 1*11. with funding from the US Office of? • 
EducatrQn and Southern Illinois University, and then a^t SU Louis with- 
fj^nding first from the USOE and then N*IE. The Director of\CSMP from 1965 to 
1979 was Burt Kaufman, and the curriculxim is in l^rge measure a tribute to his 
energy and dedication-^. Frederique Papy brought many new ideas to CSMP during 
her time as.Associate Directt)r for Research and Development; her i^nfluence 
pervades the entire curriculum.' 

■ ■ . 

The CSMP icurriccrfum is the result of a long process of devejopment, field 
testing, and revision in a wide range of geographical locations. Those sites 
varied in size and S£S of community, and students* ab41 ity 'levels, as 
meansured by standardized achievement tests, ranged from very low ability, 
including TitTe I, to upper track students defined as gifted by their 
districts. During those trials, complete sets* of ^Teacher's Guides and 
students books were written for each grade level. Materials, like the stor^ 
books^the Papy Mini-computer and analysis cards for the String Game, were 
also provided. ' ^ . 

Over the ten years of the project's history a four cycle model of 
materials development took place, essentially by grade levels: 

} . 

r. CSMP staff wrote lesion's and taught them informal 1 y^'i n local [cTas'ses.' 

2. a local pilot test version of the curriculum was prepared fro| the 
revised lessons, taught by a few local teachers and observed |^ the • 
CSMP' staff. 
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3. A "Final Expf^rimentaT' version of the curriculum, based on a revision 
of the iDcal pilot vension was taught for two years in the nationwide 
set of schools electing to pilot the curriculum* 



l^tinc 



4* A publication edition was prepared based on final revisions resulting 
from the extended pilot trials. 



A formal evaluation mechanism was established at CEMREL, which was. 
structurally independent of CSMP and funded under separate contracts. This 
evaluation group, which came to be known as th6 Mathematics Research and 
Evaluation Studies Project (MRES) was responsible for conducting summative 
evalation based 'on the Extended Pilot Tests. 



The following description of the CSMP curriculum has been excerpted from 
materials prepared by •the developer for promotional plirposes: 



DESCRIPTION .An underlying assumption of the CSMP curricu.lum is that children 
can learn and can* enjoy learning much , more math than they do now. Unlike, most 
modern programs, the content is 'presented not as an artificial structure 
external to the experience of children,^ but rather as an extension of 
experiences children have encountered in their development, both at the 
real-life and fantasy levels. Using a "pedagogy of situation;" children are 
led through sequences of problem-solving experiences presented in game-like 
and story settings. It is CSMP's strong convicion that mathematics is a 
unified whole and should be learned as such. Consequently, .thp. content is 
completejy sequenced in spiral form"^o that each student is brought i.nto 
contact with ea,ch area of content continuously throu.ghout the program while 
building interlocking , experiences of increasing sophistication as the 
situations become more challenging.. A feature unique to CSMP is the use of, 
three nonverbal languages that'give chi Idren" immediate .access to mathematical 
ideas. and methods necessary not only for solving problems, but also for 
continually expanding their understanding of the mathematical concepts 
themselves. Through these languages^ the curriculum acts as a vehicle that 
engages children immediately and naturally with the content of mathematics and 
its applications without cumbersome linguistic prerequisites. These languages 
include: the Language of Strongs (brightly colored strongs and 'dots that deal 
with the fundamentally useful and important mathematical notion pf sets'); the 
Language of Arrows (colored arrows between pairs of dots that stimulate . 
thinking about relations between objects); and the Language of the*Papy 




Minicomputer. The Minicomputer, a simple abacus that models the positi^d'r}^ 
structure of the numeration system, is used both as a computing device 
ntotivation"^ for mental arithmetic. Its language can be used to represent alT 
decimal numbers, positive or negative, Snd encourages'creative thinking about 
the nature and properties of numbers. CSMP'is flexible enought to facilitate 
whole-group,. smal 1 -group, and personalized instruction, and is appropriate for 
all children from the "gifted" to the "slow learners." It recognizes the 
importance of affective as well as cognitive concerns and has been developed 
an^J extensively tested in classrooms nationally. 

IMF^LEMENTATION REQUIREMENTS School system signs cooperative agreement with 
CSMP and appoints .local coordinator who undergoes 3-10 days of training 
(depending on* highest grade level adopted) in St. Louis during spring or 
summer prior to first'year of impl'ementaion. Coordinator trains all teachers 
new to CSMP before start of school. Smallest adoption unity's one teacher in 
o^ne classroom. ,No training charge, but system pays expenses of attending 
training. Teachers and coordinators are required to buy training kits: K-3, 
$10, 4-6 $10. Optional adopter-si.te training is available; there is a fee for 
this service. - ^ . 

FINANCIAL REQUIREMENTS 'Start-up costs for one teacher and 30 students: 
Kindergarten, $140; first grade, $T50; second grade, $200^; fourth grade, $220; 
fifth grade, $220. Replaceffient costs for 10 students: kindergarten, $10; 
first grade, $26; second grade, ^34; third grade, $47; fourth grade, $47; 
'fifth gy^ade, $48. ^ y y 

It i's clear .from the above -that CSMP is q.u1te (different from most 
traditional programs. The innovative program characteristics of CSMP form^ a 
double-edged sword With great educational potential but also with 

r " 

corresponding problems in imp^lementation, independent of students' 
achievement. Some of these are. the following: 



a. . The curriculum contains new math'fematical content particulary in 

prtDbabil ity, statistics, geometry, and other areas of ^applied 
mathematicSjv and this content is often new -to the teachers themselves. 

b. The curriculum contains new pedagogical techniques, especially the 
"languages" of the minicomputer, arrow^ diagraI^ls, and string pictures,, 

" blended int© a detailed and integrated sequence of lessons. These 
techniques require a considerable'perservice training pro-am for 
teachers, to learn how to use them effectively, and they make the 
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curriculum more complex to explain to parents and schoql 
administrators. 

CSMP often use^ stories or situations in a disrcovery approach so that 
lessons'tend to be longer than mathematics lessons -usual ly are and 
place more emphasis on whole-group discussion. This requires more 
daily preparation for the teacher, at least the first time through the 
materials, and possible a change in teaching style. 
CSMP places great emphasis on he spiral approach. This requires, as 
an article of faith on the part of the teacher, the acceptance of the 
premise that in many less'ons some students will not master all of the 
lesson on thai: day, but will Eventually learn the materiall if undue 
pressure on immediate mastery is avoided. This is in direct contrast 
to the pressures put on schools for m^astery learning, criterion, 
referenced testing, and minimum competencies — all of which mandate 
absolute levels of achievement at fixed points in time regardless of 
the curriculum being used. The use of the CSMP curriculum in the 
context of these externally mandated requirements may reiquire 
considerable skill on he part of local -administrators , as well as 
faith ^thiit this approach will eventually lead to satisfactory 
achievement. ^ ^ - * . 

Because of t.he highly integrated and interdependent nature of lessons^ 
within a yeai^ and the content froiti year-to-year, the CSMP e.lementary 
curriculum cannot be implemented on a casual basis without classes 
eventually getting far behind jn the recommended sched^jle. .ThfV means 
that, logistically, it is difficult to maintain CSMP if the le^el of 
implementation drops below a certain-minimum, and that it is 
particularly important that all CSMP teachers in a school, follow, at 
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least m.inimall.y, the intended curriculom. ^ (It also raises qu 
concerning the .transfer of individual students into and' out of the 
program. ) • ^ 

f. XSMP is somewhat more expensive in yearly replacement costs than most 
regular textbook programs. Though various monies are available for 
implementation assistance, knowledge of their availabil ity >and. % 
requirements for approval (inclcfding the access testate approved 
lists) can become difficult for school administrators. 

The data reported here are from districts and classes which for the most 
part were able to take advantage of these'' innovative characteristics and where 
CSMP was* clearly the main source of mathematics instruction, albelf with^ 
countless adaptations. The implementaton issues describe above are reported 
more fully in report 9-A-?. 

' The Evaluation of CSMP:. Goals. and ?lans ^ * ^ 

. The evaluation of CSMP has been carried out by an organizationally 
'separate division of CEMREL called Mathematics Research and Evaluation Studies 
(MRES). This group operated independently from the CSMP developers under the 
guidance of an Advisory Panel composed of five na.tionally recognized experts 
in 'the fields "of math evaluation aad education (see Appendix D). The MRES 
group produced an extensive Evaluation Report Series, 50 volumes in all, 
dealing with a wide variety of topics concerning CSMP. Th^se arfe lasted on 
pages iii and iv. 





In the first evaluation report in this series"(Evaluation Repor 
^ ^ r ' ' ^ . , 

•Overview, Design and. Instrumentation )) the goals of the evaluation of CSMP 

-were spelled out. Essentially three major issues were to be addressed. ' . 

(1) Intrinsic Merit , In the opinion of qualified reviewers, is the CSMP 
program and its mathematical content sound and relevant? 

(2) Practical ity . In terms of comparative costs, supervisory and 
instructional personnel requirements, management of students, etc., can school 
systems adopt the program with relative ease? 

*« 

. . (3) Outcomes . Do students learn the traditional skills and concepts, and 
the special skills and concepts of the CSMP curriculum, and can they transfer 
these skills and concepts more effectively (than^Non-CSMP'students) to 

4 ' \ 

unfamiliar mathematical contexts? ' 

The i^ssue of intrinsic merit was addressed early in the evaluation and 
reported in Evaluation Report l-A-2, External Review of CSMP Materials . The 
issue of practicality was addressed each year of the. evaluation and reported 
at various points in the Evaluation Report Series (see titles numbered n-C-n, 
page iv). A draft of a summary of these and other findings was prepared in 
November of 1982 and appears as Evaluation Report 9-A-2, Summary of * 
Implementation Data, Draft Report . The outcomes issue was addressed each year 
of the evaluation and reported at various points in the Evaluation Report 
Series (see titles numbered n-B-n, page iv). This report is th0 draft of the 
summary of these reports on outcomes. / 
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The main thrust of the tes'ting of student outcomes was coordinat|^' .v^f; 
the Extended Pilot Test (E.P.T,) of tbe^Experimental (Version ^of the CSMP 
curriculum. That version became a^ilable usually orje grade Tevel at a time 

at-4nter^al-s-of^a-year -or-two^ -DuringJihe. f Jnsi yea^ of the/E.H.T. at any one^ 

grade level, preliminary testing took place ^itW about a dozerv intact CSMP , 
classes and usually two or three dozen individual CSMP students and a 
comparable number of Non-CSMP classes and students from school districts close 
to St. Louis. The main purpose of the preliminary testing was to pilot the 
test instruments themselves. Then in the second year of the E^P.T. at any one 
grade level, the revised test instruments were used to test larger numbers of 
CSMP and Non-CSMP classes from school districts throughout the United States. 

The conten^t of the tests initially matched the three aspects of the 
* 

"outcomes" evaluation goal; At the beginning, the testing involved standard 
content, CSMP; content, and novel content. The CSMP curriculum dtffers ♦ 
Significantly eoough from the traditional curricula to make comparisons of 
achievement' on content specific to either CSMP or Non-CSMP rather 
meaningless^. As time went on, it became apparent that the most- valid test and 
^comparison of real student achievement involved 'content unfamiliar to either 
y group of students. Hence, the imp6rtance and utility of the MANS Tests, 
(novel content) increased. 

Figure 1 shows the schedule of testing in the Extended Pilot Test. The 
entry "Individual" in Figure 1 refers to tests which were adm'ini stererf^,, 
individually to stud'ents and which contained content novel to both groups. 
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Year 
1974 



• Figure 1 ' '' * 

Schedule of Actual Testing Indicating Test Content By Year and By Grade 

(Report Nurhberk in .Parentheses) 



1975 



•1975 



sir fwfpnl (Ift?) 



i 



Hov^l Content ^ (2BJ) 
Standard f«yi KBU 



CVr Content (281 )|cyr Content (}«t) 

fo 

V 



J977 



Individual (391) 
IWtS (JBI) 
StmdirtfContftit (Wl) 



Standard Content URD 
CSHP Content (JBI) 



1978 



Individual 



(«3) 
MB?) 



Standard Content (<Bt) 



Individual 



(5B?) 
(5B1) 



1979 



inns 



<6BI| 



IWIS 



<6Be)>WIS 



I9fin 



1981 



1982 



Bevljlon of HWS feltl to miU It avallible as a practical 
Inte9rated series of tests, tflth ei«pfiails on problem solving. 
^Aalnlstered In aradef on a cooperative bills with 
Interested uhooJ districts and In special settlnqs of 
Interest (gifted stwdtn tsl. 



Individual (eet) 
H««S (681) 



cyr Grads (m) 



HAMS (9BI) 
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' ■ * /' II. THE MANS TESTS 

Th? MANS 7ests> {-Mathematics Applied to Novel Situations) are a collection 
of short tests that assess how well students in^ grades 2 to 6 can use * 
mathematical thinking and skills to solve problems that are new or unfamiliar 
to them. . . ... 

» 

The Mi\NS Tests are normal ly contained in two student booklets at each - 
^ grade level. The booklets contain several t^ts. .Each test has its own 
directions which a specially trained tester follows in explaining ^he task and 
the sample items. Students/ then complete the items in that.test on their 
own. A flexible time lim^t, usually about 5 or ^6 minutes, a-llows almost all 
students to^ finish. 

^ach MANS Test takes up one or two pages »in a booklet so that diapgrams and 
illustrations are large, words are easy to read and there 'is ample space for 
students ti^ *do scratch w^rk. For most tests, students produce their own 
answers instead of selecting one of several given alternatives; Answers are 
to be written in the booklet and can be erased or (grossed out; no special 
pencil is required. ^ ^ ' 

The MANS Tests wer^ developed by the MRES staff to evaluate CSMP. The 
MANS Jests ^evolved from a^ heed to develop te'sts which could be used to compare 
the progress of CSMP and Non-CSMP students. The tests had. to reflect the 
emphasis CSMP places on generalized thinking skills and problem solving. 
However, to be fair to 'Non-CSMP students, they could not/contain any of the 
representational languages or activities associated with CSMP. 
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To meet the peed for tests that could be used .to evaluate %he effectivness 
of CSMP as a program that develops mathematical thinking and problenf solving 
skills, an extensive process of test development and revision was undertaken. 
The development occurred .sequentially, on4^ grade level at a time. At each 
grade level, the Mathematics ReSearcffand Evaluation Studies (MRES) staff 
first developed^ protptype tests. Sometimes* the ideas for the tests were 
adapted from-ideas in previous' research in mathematics education; most times . 
the ideas were original. Th6 Advisory Panel (whose members are listed on the 

xfront -cover) independently reviewed all of the test projtotypes.* Occasionally, 
teachers, math supervisors', and researchers also reviewed the tests. If the 
tests survived these reviews, 'they were pilot tested' in a few local classes. . 

*0n the basi^s of results from these pilot classes, tests were revised, in some 
cases%liminated. 

^ The' original version of t'he MAMS Tests resulted from this (5onti^5jufpg 
process of* de\fe>T5pment, re.vijgw, testing, and revisiopl. This version was* used 
in the first Extended Pilot Test involving 15 to 20 local classes/ After 
further refinement of the tests, they were used in the second BUended Pilot . 
""Test at that grade level. This evaluation study involved fron^ \o to 60 
■^^classes ia several states, an? is the basis for the main data o^Ws report. 

' .. ■. ^ 

At^each stage in this process of development, review, testing, and 
revision^ the work was guided by the Advisory Panel. Some of the important 
considerations in the review and revisions of these tests, were the following: 

. Intrinsic Meri.t: importance of the mathematical skill required; 

^curricular fairness; student interest in the novel problem context, 

- \ ■ 
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Mm ini strati on: clarity and brevity of directions; student . fyjlml^-^^^^ 



1/ * • 

understanding of the task; low reading level; attractive format; 
unspeeded. 



Technical: item analy'Sis including range of difficulty level's, error 
analysis, discrimination coefficients; test analysis ihclLid\ng 
« ceiling and floor effects, ability level differences, reliab^ility 
(KR 20 studies of internal consistency), analysis of class me^s 
'and evidence of construct validity/ 
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III. THE EXTENDED PILOT TESTS: MAIN RESULTS .F;:\n:> n 

, The mam results to be reported here are those obtained in the second year 
of theE.P.T. at each of grades .two through six. This report will concentrate 
on the CSMP/Non-CSMP comparisons of student achievement on standard and novel 
content. . ( 

' I . ',' ■. . ■ 

The main resu.lts reported here h^vethe following characteristics. The'y 
comparfe the performanee of CSMP students to that of Non-CSMP students of 
. similar ability. They are based on results obtained from tests tha't ar6 
technically ^ound. Tliey are based on' the performance of CSMP and Non-CSMP 
stu{]ents who (by and. large) have htrd at least three years of formal 
instruction in CSMP or a more traditional program, respectively'^ and who are 
eight or more years of age. They are based on classroom experiences which are 
close to typical for both CSMP and Non-CSMP. Finally, for each' grade level 
the data is collected in a similar fashion. 

Design. S.chool districts have used many strategies in adopting CSMP. One 
strategy n^er adopted is the kind of random assignment of teachers and 
students to experimental (CSMP) and control classes that would satisfy the 
conditions required for a true experimental comparison. Thus, the design- -of 
the Extended Pilot Test has had to be less rigrous. Generally, the strategy 
has been to first select a representative group of CSMP classes from districts v 
using CSMP at that. qr0e level. Then, in Gooperation with local districts, 
appropriate NON-CSMP classes were selected in such a way as to minim'ize as 
much as possible factors other than curriculum which might cause differences 
in achievement between the two groups. 
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Through third grade, CSMP classes were selected from the ^ocalL^Jj^ggt^| 
jirea and'from districts with several CSMP classes (i.e.^, ^it least six). 
Comparison .classes werfe usually selected from the same schools as CSMP 
classes, with teachers and students as similar to their CSMP counterparts as 
fJosslble* ' ' , 

jf:rom fourth'to. sixth grade, c school and district wide adoption of CSMP 
became more common and comparisbn classes came to be selected from other, 
similar schools in the district^ or from other districts using CSMP at lower 
grade levels. Most' districts with at least tw CSMP clas,ses were included in 
these later E.P.T.'s. ^ , ' 

An average of 50% of'the available CSMP classes participated in these 
Extended Pilot Tests. The curriculum us^d by the comparison classes was ttie- 
usual mathematics- ctirriculum for the district, almost always one of the common 
elementary textbooks. They were only minor differences between these 
textbooks, and in the analysis of class means, these comparison classes were 
combined into a single "Non-CSMP" category. 

Table 1 gives the distribution of classes, participating in the E.P.T. 

4 

second year testing^, by sites and grade level. ..During the five years covered 
in this testing, over 300 classes (half CSMP and half Non-CSMP) participated 
from 24 districts (sites). Very few of these 300 classes weVe in fact the 
same class counted twice because there were two IZ-month gaps in the 
one-grade-per-year schedule of the E.P.T. and because the ^participating 
districts varied somewhat from year to year. With the exception of some 
classes in sixth grade, all the classes were regular in that they hadn't be$n 
grouped by ability. However, as can be seen by the last line in Taljle 1, t;he 
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^ Distribution of Classes Participating in the E.P.T. Testing • r/u\l[p^|j 
(Number of CSMP classes given first, Non-^SMP second) 



Site 


Type of 


Section of . 


Second 


Third 


Fourth 


Fi fth 


Sixth 


No. 


Communi ty 


Country ,\ 


' Grade 


Grade 


Grade 


Grade 


Grade 


1 


Medium City 


north Central 


• 




• 


0-3 


0-6 


2 


Exurban 


Central East 






2-2 , 


2-2 . 


2-2 


3 


Medium City 


South Central 


6-6 


• 




• 




4 


Large City 


North Central 


6-6 




■i 
- 


• 




5 


Large City 


Central East 






- 


• 


< 

2-2 


•6 


Suburb \ 


Central 


3-3 


4-5 - 


1-3 


0-2 


- 


7 


Small City 


^West 








0-5 




0 

o 


jUuU (U 


riorxn centra i » 










1 Cit 
1 -0 


9 


Subgrb 


North East 










0-4 


10 


Suburb 


North East 






3-3 






n 


Small City 


North Central 










7-0* 




Suburb 


i Central . 


3-2 


4-4 


6-6 


6-6 


8-6 




Large City 


C'entral 










0-6* 


14 


Medium City 


North Central 










0-5* 


15 


Suburb 


Central 




** 


6-0 


6-0 




16 


Small City 


North Central 


1-1 








• 


17 


Suburb 


North East 


2-2 




6-0 


6-0 


6-0 


18 


Large City 


South 








5-3 


• 


19 


Surburb 


Central 


6-5 


. 4-3 ^ 


2-2 


2-2 




20 


Large City 


Central East 






3-4 






21 


Smal 1 Town 


South East 


- 


15-12 


- 




- 


22 


Medium City 


North East 


6-6 


■ 6-12 




- 


- 


23 


Large City 


Central 


3-3 ^ 




1-1 


1-2 




24 , 


Suburb 


Central 








3-0 




TOTAL CLASSES 




.36-34 


33-36 . 


30-21 


31-25 


26-37 


Percentile Rank on 


V 

Covariate 


56-54 


55-56 


64-62 . 


61-60 


77-78 


*Upper track classes, "i.e. students 


specially grouped by ability 







Graces 



0-9. 
6-6 
6-6 

6- 6 

2- 2 
8-13 

0- 5 

1- 6 
0-4- 

3- 3 

7- 0 
27*24 

0-6 

0- 5 
12-0 

1- 1 
2Gr2 

5t3 
14412 

3-4 
15-12 
12-18 

5-6 

3-0 




rn 




classes tended to be somewhat above average in ability; this was pa 
true in sixth grade due to the presence of upper track^ classes. 'At each grade 
level, approximately 66 clasps were tested, about 30 CSMP and 30 Non-CSMP.. 
Again, from the last line of Table 1 , , the^abil ity level of the CSMP classes 
matched^that of the Non-CSf^P classes at each^grade.*^ Finally, by scanning the 
second and third columns of Table 1, the reader can see that nearly every type 
of community and section of the country is represented. Rural districts and 
the Far West part of the country are not well represented, however. v. 



Categories of MANS Tests . Altoge^ther some 111 tests were given during the 
five years of testing reported here. Appendix A gives a-description of these 
tests. Many of the tests were used at more than one grade level, with items 
of different levels of difficulty; jonl*y 54 truly 'different tests are described. 

For purposes of reporting, the tests havis been categorized. Ninety-five 
of the 111 tests emphasize one of seven mathematical processes. These seven 
processes are named and described on the following pages and an example 
showing a few itemsj^om a test in that process category is given. 

The other 16 of the 111 tests emphasize one of five special topics in \ 
mathem^ics: Algebra, Geometry, Logic, Organizing and Interpreting Data, and 
Probability. Again, examples of each can be found i-n Appendix A. These 
special topics were tested mostly in grade -six but to some extent in grades 
five and four. 
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Computation 

StralghtfoQ^ard calculation 

with basic facts and algorithms. . 

Examples from a third grade test. 



SUBTRACTION 

V 

U 73 
-5 -5 



64 
28 



HULTIPLICATION 
5x8- 



31 
x2 



Estimation 

Rapid calculation of 
approximate answers. For 
this category only, fixed 
time limits are prescribed. 

Examples ^rom a sixth grade 
test. Short time limit. 



pit 



44- 3 



/ 



CHCa CMC 
Ejcictly 1 




Mental Arithmetic 

Solution of numerical 
problems that emphasize 
an understanding of numbers 
and operations, btit do not 
require great mental . 
computational facility. 

Examples from 'a sixth grade 
test. No "Iscratch .work" is 
allowed; 



7,001' - 8.999 • 



9I.0C1 - 



— 250 • ISO 



]• ss.oop 



5,000 



J2 X 500 - j ( 



101 X • 



7 X « - i 301 



14 X U 



CD 



19 



9 • 



Number Representations > 

Recognition, or prodution ' ^ 

of different ways of , ' 

representing numbers/ • p_ 

including place" value^ - * * / \ 
number line?, measurement. ^ ^ = 

Example from a. second grade ] »w m6? 

test For each of the first " '\ 

group of items jA through F), 

the t-ester says" aloud a number. »iu fti«btri» lo ■ortwunic?? 

for students to write in the 

blank. 

»ut AuMr it 100 ■Oft th«n «17 



Relationships and Number Patterns 

Recognition or application of 
given patterns, orders, or 
relationships in sets of numbers. 

Examples from a fourth grade test. 
Three sample items, explaining how 
the ^'machines" vork, are done 
previously. m 



Q 




Elucidation of Mutiple Answers 

Production of many correct answers hiU%: iU9 otit tar- mh. 
to a given problem. v/ \ / 

M u ftt « tats; scon. jiL 100 

Examples from a sixth grade test. 



l^ltt; Vm fiMbtrt Mt bt bttM«i SOO and MO. 

1W of tha digits Mt bt f. 
tlvt sn tilt corrtct sMwtfs. 



2u . 
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Word Problems 

Solution of word problems' 
requiring low levels of 
reading comprehension and t 
computation and classified 
according to type of problem. 

Examples from a second grade 
test. Tester reads the items, 
aloud t ^rame-^y-frame. 



A fintMtlc 11^ Afttr OM tfiy mt v 

*»ttrt1ftf*tnf, hit 9oni 2 ■tlti. 

Method of Analysis . Each of the participating classes was given some test 
of general academic ability. In the earlier grades, the actual test used ' 
varied from site to siteand year to year but was usually either an academic 
intelligence test o^ a test of reading comprehension or vocabulary. In the 
later grades, classes- unifo.rmly took an appropriate level of the 
Gates-McGinitie Vocabulary Test. 

For the main results reported here, the analysis of student achievement 

was done each year on class- means. F^r each test, a mean score was -calculated,^ 

across all the students in the class who took all the achievement tests in the 

* 1 
battery and who also took the test of general ability. The corresponding 

mean score on the test of general ability was also calculated. In both cases 

raw scores were used. Then an analysis of covariance procedure (with genera] 




ability as the^covariate) was used on each test to compare the mearv^^^'AJp* 
the CSMP classes that year versus the mean score for the comparable Non-CSMP " 

• • • 

classes. From- this analysi&,sof covariance can be determined the probabil ity>' 
that the observed difference in mean scores could have occurred by chance. 
(If that probability is less than 5%, the difference is often deemed to be 
"statistical ly significant". ) y 

For each of these' achievement tests, the -mean scores for CSMP and Non-CSMP 
classes were compared in another and much simple way: the percentage 
difference in mean scores between the two groups of classes. (Previous to 
this the mean raw scores had been statistically adjusted to take into account 
differences between the two grcfups'^on the test of general a^jilit/. Since 
, these two groups were invariably quite simvlar in general ability, such 
adjustments usually were quite small— less than 2%.) 

Results By Grade and Test Category . In Table 2 there is a summary of the 
results of the te^ti-ng, by grade level. For each grade, the table gives the 

* * 

number of achievement tests that were administered, the number of tests for 

which the analysis of covariance indicated a statistically significant 

''difference in favor of CSMP or Non-CSMP, and the average percent difference in 

the adjusted mean scores. The bottom line of the table shows that 111 tests 

were administered over all grades. Of those 111 tests, 69 produced a 

# 

statisticalTvy significant difference in favor of CSMP, two in favor of 
Non-CSMP, "and an average percent difference in the adjusted mean scores of 
13.4 in favor of CSMP. " 



S 



No'rmally less than 15% of the students in a class had to be excluded due 
to this requirement. In addition, normally a class lost another 5% due to 
students joining the class late in the year or being labeled "special" in 
some, handicapping way. 
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Table 2 

Grade Level Summary of E.P.T. Test Results 
(All Categories Combined) 



Grade Level 



Number of 
Tests 



Number of Times 
Significant 



Average 
Percent 



In 
F.avor 





of E.P.T. 


Administered 


CSMP 


Non-CSMP 


Difference 


of: 




Second Grade 


11 


3 


0 


13.4" 


CSMP 


• 


Third Grade 


15 


10 


-e 


12.9 


CSMP 




Fourth Grade 


25 


18 


1 


18.9 


CSMP 




Fifth Grade 


26 


15 


0 


12.9 


CSMP 




Sixth Grade 


34 


23 


1 


10.1 


CSMP 


• 






t 








) 


Total Across 


ni 


69 


2 


13.4 ^ 


CSMP 



All Grades 

The results at each grade reflect the overall results rather consistently 
with two minor exceptions: in second grade, not as large a proportion of the 
tests produced statistically significant differences in favor of CSMP, and in 
fourth grade the average percent difference in adjusted mean scores was much 
larger. c 



In Table 3 a summary similar to that of Table 2 isv given according to test 
category. 



■ ). 
i) * 



\ 
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Table 3 

Summary of E.P.T. Test Results 
- All Grades Combined 
Given by Test Category 



Test Category 



Number of 
Tests 
Administered 



Mathematical Processes^: 

Computation 8 

Estimation 13 

Mental Arithmetic » 21 

Number Representation 12 

Relationships and 

Number Patterns 22 

Elucidation ^ 6 

Word Problems 13 

Special Topics: 

Algebra 4 

Geometry 2 

Logic 2 

Organizing- and 

Interpreting Data 3 



ll.umber.of Times 
Significant 
v.CSMP- Non-CSMP 



1 

9 

19 
5 

16 



8 

0 
0 



1 

0 
0 
0. 



0 
0 

0 
V 
0 




Prebabil ity 



3.8 
8.5 
19.1 
8.6 

20.8 

16.7 
15.1 

11.0 
3.0 
0.0 

o:3, 

11 .2 




Average In 
Percent Favor 
Difference - of 



CSMP 
CSMP 
CSMP 
CSMP 

CSMP 

cSmp 

CSMP 

CSMP 
Non-CSMP 



Non-CSMP 



CSMP 



Total Across 
All Cate'gories 



111 



69 



13.4 



CSMP 



V 



24 



Most of the tests were in one of the Mathematical Process cate^ovfieV. 
those, the results were strongly in favor of CSMP in Mental " Arithmetic and 
Relationships and Number Patterns . In those two cases the CSMP advantage 
showed up 'in both the number of times the tests produced statistically 
significant differences and in the average percent difference. The results 
were almost as strongly in favor of CSMP in Elucid'^tion and Word Problems, and 
« slightly in favor of CSMP in Estimation and Number Representations . In the 
Computation category there was essentially no overall advantage to either 
group. ; ' ^ 

In terms of Special Topic categories, the reader is reminded that the 
majority of these tests were given only in sixth grade. Only in two 
categories ( Algebra and Probability ) was there, strong evidence for a 
difference, and in both cases it was in favor of CSMP. 

Graphs of Class Means . On the following- pages, graphs of class means for 
Total MANS are shown for each of grades four through six tp illustrate the ■ 
data of Table 2. For each graph,' an entry on the graph represents a class; 
/ the position on the graph is determined by its "ability" score (IQ, reading, 
vocabulary--whatever happened to be used that year) and by its Total MANS' 
score. 

CSMP classes are shown byx's, Non-CSMP classes by m's. A regression line 
has been drawn in each graph to show 'the best prediction of MANS score for a 
given ability score. One does not need a test of significance to determine 
that there is a clear pattern for CSMP classes to be above the regression line 
and Non-CSMP classes to be below the line. 
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Figure 2, Fourth Grade Class Means / 
(x=CSMP class, ^=Non-CSMP class) 




I.y._^...THE EXTENDED PII^ TEST: SECONDARY RESULTS 



Note to the reader: T})is chapter has not been completed for this draft 

report Some illustrative data from each section is given 



below. 



Analysis of School and District Data 



The graphs shown on pages 26 and 27 show entries for each class. In each 
case, the CSMP classes are doing better than Non-CSMP classes and the 
differences are statistically significant. When the data are aggregated by, 
schools, instead of classes, the "differences remain significant, but the 
pattern of mean scores becomes clearer. 



Notice in Figure 4 that there are six CSMP classes which are scoring well 
below the regression line. Figure 5, below, is the graph of school means; the^ 
six low scoring classes were concentratfeti in two or three schools rather than 
b^ing scattered randomly. 



Total flANS 




Vocabulary 
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Figure^S. Sixth Grade School Means 
•(x=CSMP School , •=Non'-CSMP School) 
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^Figure 5 illustrates two pointL First, at higher grades it is school . 
rather than teacher that may be the more effective factor in achievement* 
This may be because the MANS Tests are cumulative in nature rather than 
testing specific sixth grade content, because of the influence of the ^ 
principal, oY* because there is cohesion among the teachers iriAhow to teach 
the math curriculum. ^ The second point is that the CSMP effect is more evident 
in school level analysis; in Figure 5 there were 16 CSMP schools, 14 of them 
did better than all but one Non-CSMP schools while the other two did worse 
^han all Non-CSMP schools. 

Analysis of district ,leve-l means proM4cles similar findings — still 
significant differences and clearer interpretations. 

Analysis^ of Item Data 

Extensive item analyses werie done each year ^n all items in the MANS 
Tests. There were several kinds of items which produced large differences 
year after year between CSMP and Non-CSMP students. For example CSMP "^^udents 
consistently did better than Non-CSMP students in multiplication of whole 
numbers,' fractions, and decimals. THis advantage was largest in the years in 
which these topics were being introduced, in mathematics and reflected the 
early CSMP emphasfts. The differences decreased in later grades. Similarly 
CSMP students did noit initially do as well in whole number subtraction and 
long division, again reflecting the later (or lesser) emphasis of these topics 
in the CSMP curriculum. ^ 
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Analysis of Student Level Data ^ ■ 

In order to analyze the effect of .the CSMP curriculum according to ability 
level of student, the following procedure was followed at each grade level. ' 
Students were grouped according to percentile- rank 'on the covariate 
test--reading, vocabulary, or IQ. There were four groups corresponding to 
highest quarter in ability, second highest, etc. Then comparisons on MANS 
Tests between CSMP and Non-CSMP students were niade for each ability- quartile. 

Figures 6 and 7, from sixth grade, illustrate the different results which 
were found in the categories Number Representations and Elucidation (both of 
which had significant differences in favor of CSMP in Analysis of Covariance 
on class means J 



Category Total 

Number Representations 




Vocabulary Score 

Fi^ui^e 6, Scores on Number Representations by Ability Level of Students 
(x=CSMP group of students ,# =Non-CSMP) 
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Vocabulary Score 



Figure 7, Scores on Elucidation by Ability Level, of Students 
(x=CSMP grouping of students, =Non-CS'MP) 



* On Elucidation , the low ability CSMP students had just as large a 
superiority over their non-CSMP counterparts as did -the high ability 
students. This is what happened most of the time, i.e. the results were 
consistent at different ability levels. But on Number Representations , the 
low ability CSMP students did not share in the otherwise consistent advantage 
of CSMP students. This result happened occasionally,' most often in the less 
hard tests; such as Computation. 



3, 
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Appendix A 

Description of MANS Te^ts 
^ Comprehensi ve List 



This is. the comprehensive list for the 
most current versions "^of the MANS Tests, 
There are minor descrepencies between these 
tests and the ones used in the E,P,T, 



Process Categories: 

C: Cornputation 

E: Estimation • ^ 

M: Mental Arithmetic 

N: Number Representations 

R: Relations i Number Patterns 

U: Elucidation 

W: Word Problems 



I 

Special Topic Categories: 

A: Algebra 

G: Geometry 

L: Logic 

' 0: Organization of Data 

P: Probability 



\ 
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Category C: Computation 



01 Whole Number Computation 

Abstract: Given straightforward computation problems involving whole 
numbers, prpduce exact answers (by calculating on paper if 

, necessary). The items do not have the multiple choice 

response format but are simTTar in range and difficulty to 
those found in the standardized achievement tests of the appro- 
priate grade level. 

Grade Leve"f)5: 2, 3, ,4, 5, 6 ' ^ 

Examples (from Grade 4): 352 675 143 

^^683 -469 - x 5 67492" 

C2 Fraction Computation 

Abstract: Given straightforward computation items involving simple 

fractions, produce exact answers (by calculating on paper if 
necessary). Though the items do not have the multiploi^hoice 
response format, they are similar in range and difficijlty to 
those found in the standardized achievement tests of the appro- 
priate grade level. 

Grade Levels: 4, 5, 6 

Examples (from Grade 5): 



' C3 Decimal Computation 



Abstract: Given straightforward computation items involving one and two 

place decimals, produce exact answers (by calculating on paper if 
necessary). Though the items do not^ have the multiple choice 
response format, they are similarTfT range and difficulty to 
those found in the standardized^ achievement tests of the appro- 
priate grade level . 

Grade Level : 6 

Examples: 0.5 + 0.25 = [HI S-l.S^QU 0.5 x a.5 =» | [ 
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^ Category E: Estimation 

El 2 or 5 or 10 Times 

Abstract: Given two numbers, quickly estimate whether tHe first is about 
2 or 5 or 10 times as large as the second. A sample is worked 
collectively. 

Grade Levels: 3, 4 ' • < 

Examples (from Grade 3): 65 is about times as large as 12 

98 is about times as large as 51 

E2 Estimating Intervals: Addition 

Abstract: Given a computation problem involving whole number addition, and 
5 fixed intervals (0-10, 10^50, 50-100, 100-500, 500-1000), 
^ determine which interval contains the answer to the problem, and 
put an X in the interval. By instrjJction, format and short time 
limits, students are discouraged from computing exact answers. 
Two or three sample items are done collectively. 

Grade Levels: 2, 3, 4, 5 

Examples (from Grade 2): SV + 53 0 10 50 loo 500 1000 

189 + 273 0 10 50 100 500 1000 

E3 Estimating Intervals: Subtraction 

Abstract: The scale is similar to E2 (except that it involves wtjole number 
subtraction) and follows it directly in the test booklets. 

Grade Levels: 2, 3, 4 
^Examples (from Grade 3): 93 - 86 0 10 50 TOO 500 1000 

147 - 99 " 0 10 ■ 50 100 500 1 000 . 

♦ 

E4 Estimating Intervals: Multiplication 

Abstract: The scale is similar to E2 and E3 (but is devoted to multiplica- 
tion with whole numbers for the most part) and follows them in 
the test booklets. 

Grade Levels: 2, 3, 4, 5, 6 

Examples (from Grade 4): 40 x 10 0 10 50 100 500 1 000 

. 4 x 29 0 10 50 100 500 1000 



E5 Estimating Intervals: Division 

> Abstract: The scale is similar to E2, E3 and E4 (but is devoted to division 
with whole numbers for the most part) There are only four fixed 
intervals (0-1, 1-10, 10-20, 20-100) in the response format. It 
follows E4 in the test booklets. 

Grade Level: 5, 6 

Examples:' 1 J 15 0 1 10 20 100 

101 * 9 0 1 10 20 100 



E6 Estimating Fractions <, *, >1 
. Abstract: Given a calculation ( +, -, or :) of two numbers (at least one of 
which Is a fraction or mixed number), quickly estimate whether 
the answer would be less than, equal to or more than 1. Students 
are encouraged to work quickly and not to compute exact answers 
be-ftjre making their choices. A completed sample item Is 
provided'. 



Grade Level : 6 



CHECK ONE 







Less' than 1 


Exactly 1 


More than 1 


'1- 


1 

T28 








4- 


3 









Category M: Mental Arithmetic 

r 

Ml Whole Number Open Sentences 

^ Abstract: Given an open sentence, where the box may be either on the right 
or the left of the equal sign, where the numbers are large and 
easy to work with, and where only one operation is used, put the 
number in the box which makes the sentence true. By instruction 
anOrompting, students are discouraged from "computirig the long 
way" and are not allowed to do any figuring on paper. 

6rade Levels: 2, 3, 4, 5, 6 
Examples (from Grade 3) 500 + | | = 800 

I I - 150 = 50.r 
2 X 200 = □ 

M2 Above and Below Zero ^ 

Abstract: Given a starting score (which could be above or belov^iilro) , and 
haw much the score went up or down, select the correct final 
score (multiple choice). 

Grade Levels: 2,3 

Examples (from Grade 3) 
Score at the start: 3 below zero 

Then: Lost 4 - 

Score at the end: 7 below zero 1 below zero 1 abovfe zero 7 above zero 

Score at the start: 2 above zero 
Then: Lost 4 

Score at the end: 6 below zero 2 below zero Zero 2 above zero 



M3 Negative Hits and Misses 



Abstract: Given the description of a "game" with two rules {•^) each hit 
means a gain of 5 points and b) each miss means a loss of 1 
point) and partial information on the outcome of turns, the - 
student must deduce the missing information. Two sample items 
are completed collectively. 

Grade Levels: 4, 5, 6 



Examples t 



Started with 
a score of t 



Pam: 
John: 



4 above zero 



Number Number Ended witrt 
of Hits of Misses a score of 



3 above zero 



15 below zero 



3- 1$ 



(provided, 
but not * 
mentioned in 

.instructions) 



M4 Fraction Open Sentences \ 

Abstract: Given an open sentence involving at least one fractioVi, and one 
of the four arithmetic operations, complete the sentence. 



Grade Level; 6 
Examples: ^ 



1 — 



. 3 
~ 4 



M5 Decimal Open Sentences 

Abstract: Given an open sentence involving at least one decimal number and 
one of the four arithmetic operations, complete the sentence. 



Grade Level : 6 



Examples; 



0.5 



0.75 — 



0.5 



/ 



Category N: Number Representations 



- Nl Writing >Whole Numbers 

Abstract: Part I: The student mu^t write numbers as they are read aloud 

by the tester* \ 
Part II: Given a number, wKj^tten in the test booklet, the student 
must write the number which is 1 (or 10 or 100) more 
than it. A ^sample item is worked collectively. 

Grade Level : 2 

Examples: Part I: Tester says, "Eight hundred twenty** (repeats) 

Tester says, "Seven thousand sixty five" (repeats) 
Part II: What number is 1 more tha8^999? 

What number is 10 more than 495? 



N2 1, 10, 100 or 1000 More 

Abstract: Given two numbers, decide whether the first number is about 1, 
10, 100 or 1000 more than the second number. (None is exactly 
right.) Two sample- items are worked collectively. 



Grade Level: 3 
Examples: 

4,265 is abotjt 
1,001 is aSout 



1 

10 

100 more than 4,254 
1000 

1 

10 

100 more than 998 
1000 
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N3 Constructing, Numbers 



Abstract: Given the use of only four digits (2, 5, 7 and 8) and the rule ' 
that no digit be used more than once, construct numbers like the 
smallest (or largest), the second smallesft (or largest) or the 
closest to a given number. The constructed numbers are to be of 
" either 2, 3 or 4 digits and sometimes restricted to a given 
range of numbers. Collectively, to clarify the rules, two* 
incorreict answers and the correct one are examined for two^ 
Sample problems. 

Grade Level : 4 , ' 

Examples: What is the second largest four digit number? ' 

What is the smallest three digit number between ^ 

730 and 850? ^ 



What fouV digit number between 2,000 and 3,000 is^ , 

closest to 2,800? _ 



N4 Representing Fractions \ ^ ^ 

Abstract: The scale has five short subsections each containing one of two 
kinds of items: a fraction 'or mixed number is given in standard 
form and must be represented in another specific' way or else 
that process is reversed and the response format is multiple 
choice. Instruction is largely in the form of a, written 
question or command at the beginning of each subsection. 

ft 

Grade Level: 4 

Examples: Put an arrow at inches* ■■ | » 1 1 1 1 1 1 j'l j 1 1 1 1 I j'l j » j ' j * |' 

3 In* 4 In* 5 in* 6' in* 



How much is shaded? 



N5 Representing Fractions and Decimals 



Abstract: The scale has five short subsections each containing one of two 
kinds of items: 'either a mixed number or decimal is given 
in standard form and must be represented in another specific ^ 
way or else that process is reversed and the response format 
is multiple choice* Jnstrtiction is largely in the form of a 
written question or command^at the beginning of each subsection*. 



Grade Level: 5, 6 

Examples: 'Put an arrow at 1.35 inches. 



|u M |i u i j j 1 1 1 1 1 1 i 1 1 M M I i I i i 1 1 M 1 1 M 1 [ j 



How much is shaded? 




1 

T 



1 



none of these 



(A completed sample was* given.) 

N6 Equivalent Fractions and Decimals ^ 

Abstract: Given* a fraction (or decimal) determine which members of a set of 
fractions (or decimals) are equivalent to it. A sample set 
of four completed items is shown. 

> 

Grade Level : 5, 6 » 

Examples: Circle all the- fractions that are equal to the one in the box. 



9 



4 



3 

r 



10 



Category R: Relationship & Number Patterns 



Rl Solving Number Rules 
Abstract: 



Given 3 clues^(i.e., pairs of numbers) in a game, determine what 
the secret methodMs (i.e., the unique rule relating each of the 
pairs. of numbers) and then use the rule to calculate the missing 
number from the fourth pair* 



Grade Levels: 2, 3, 4, 5, 6 
Example^s (from Grade 3): 



Maria's Game 



Jim's Game 





Class 


Maria's 


Class 


Jim's 




said: 


answer: 


said: 


answer: 


First clue:. 


5 


10 


2 


6 . 


Second clue: 


7 


' 12 


5 


9 


Third clue: 




13 


10 


14 , 


Question: 


2 


□ 


□ 


12 



R2 Usin^ Number Machines 
Abstract 



Given labelled '^number machines" in sequence^and either ^^^^^ 
initial or the terminating number, detemine the other nmb^^T^ 
There is an introduction showing that "number machines "'take in 
numbers; add, subtract, multiply or divide by a fixed quantity; 
and give out the resultant number. TheygJ^hree sample items 
(each with a "number-machine" sequence)^^ worked collectively. 



Grade Levels: 3, 4, 6 
Examples (from Grade 4): 



x5 



Q 



x2 



♦ 1 



-r2 



t) 



R3 Sequences 
Abstract: 



Given an incomplete portion of an additive sequence of numbers, 
determine the missing number. One sample item is worked 
collectively. 



Grade Level : 2 
Examples: 28, 25, 



19, 16, 13 



1, ^ li, 2, , 3, 3i^ ' 4 
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R4 Which Result is Larger 
Abstract: 



Given two quantities (usually similar computation problems 
using +, or x) mark the one which yields the larger result, 
or mark them both if they' are equal* By instruction, format and 
t.ime limits, students are discouraged from computing exact 
answers* The correct resportse should be more easily determined 
by inspection than by computation* Two sample items are worked 
collectively. 



Grade Levels: 2, 3 

Examples (from Grade 2): 585 + 250 Q 

580 + 290 n 



3 X 31 Q] 
31 X 3 □ 



R5 Labelling Number Lines • 

Abstract: Given partially labelled number lines, with varying increments, 
determine certain missing numbers* A sample item is worked 
collectively. 

Grade Levels: 2, 3, 4. 5 , 6 

• Examples (from Grade 2): 



1 



IT 



16 



i 

22 



20 



24 



-J— 
28 



32 



36 



R6 Multiplication Series 
Abstract: 



Given an incomplete portion of a multiplicative series of 
numbers, determine the donstant multiplier involved in order to 
complete the portion shorn. Portions of several series are 
shown altogether with oneV two or three numbers missing from 
each. A sample series is examined and completed collectively. 



Grade Level : 4 
Examples: 



• 


1 




\ 


1,000 


10,000 


100,000 


• 








\ 









R7 Which Fraction is Larger 

Abstract: Given two non-whole numbers written in fractional form ( a proper 
fraction, an improper fraction or a mixed number), circle the 
larger one. A completed sample item is shown. 

Grade Level : 5 > 6 

Examples: j or ^J 

% or J 

R8 Which Decimal is Larger 

« 

Abstract: Given two non-wht)le numbers written in decimal form, circle the 
larger one. A completed sample item is shown. 

Grade Level : 5,6 

/ 

Examples: . 4.999 or 5.1 
1.5 or 0.58 



R9 Fractions Between Two Others 

Abstract: Given two fractions, write another which is larger than the 
first and smaller than the second. 

Grade Level : 6 . 

Examples: is larger than 3 , but smaller than "J* 

. , _L J. 

: 1s larger than , but smaller than z 



RIO Decimals Between Two, Others 

Abstract: Given two decimal numbers, write another which is larger than the 
first and smaller than the second. 

Examples: is larger than 1.25, but-smaller than 2.0 



is larger tharr 0.42, but smaller than 0.43 



Category U: Elucidation 



U1 Number Sentences About 8 

Abstract: Students are to produce as many different "sentences about 8" as 
possible, always in the form "8 » Four correct answers to 

sirflilar exercises about 9 are examined collectively. 
(9 » 10 - 1, 9 « 1 + 5 + 6, 9 » 3 X 3, 9 « 18 - 2). 

Grade Level: 2 r 

- f " - . 

Example: My number sentences about 8. 

8 « 

8 » : 



U2 Producing Many Answers 

Abstract: Given several different situation^ each of which poses a problem 
for which there aKe many correct solutions-, produce as many of 
them as possible. For each situation, some potential solutions 
are accepted or rejected for not following the given rules as 
Inappropriate. 

Grade Level: 3, 4, 5, 6 

Examples (from Grade 3): 

Rules: Take out two balls. 

Add the two numbers to get a score. 

What are the possible scores? 6, 2. 35 



Rules: Write all the two digit numbers you can. 
^ Use only the digits 1, Zi 3. 

Give aVl^ the numbers that follow the rules. 34. 22 



U3 Getting to 12 . 

Abstract: Given a starting point (0), a goal (12) and two rules, invent as 
many ways of reaching the goal as possible. Tfie rules are that 
only the numbers 2, 3, 5 i 7 can be used along with addition,, 
subtraction, multiplication or division. Two sample solutions 
(see below) are worked collectively. 

Grade Level: 6 

Examples: 

Sample 1 : 0 +_7 » 7 

7 x_2 = 14 ^ 

Sample 2: 0 +_5 = 5 ' 

5 +_3 » 8 

8+2-4 

4 X 3 » 12 



Category W: Word Problems 



V 

Wl One Step Word .Problems 

Abstract: Solve word problems in which the "story (including the question) 
is read by the tester while the student looks at 4 series of 
cartoons and/or follows the story in the captions beneath the 
cartoons. Seven items require one-step solutions; two items 
require two. 



Grade Level: 


2 






Encamp! es: 








How many baoanas did 






she buy? 













Jill spent 6i to Bananas cost 
buy some bananas. 2i each. 




How many marbles 
did** he have to 
begin with? 



Jim found 3 And now he 

marbles but has 5 marbles, 

he lost 4. 



W2 Two Stage Word Problems 

Abstract: Solve word problems in which the solutions require two opera- 
tions. The numbers* in the problems are relatively small; the 
computational, and reading requirements are simple. 



Grade Levels: 3, 4, ,5, 6 
Examples (from Grade 4): 



Pam gets 50^ each week. 

She always spends 30^ and saves the rest- 

How much will she save in 4 weeks? 



Tom has 3^ more than Ann. 

Tom has 5^ less than John. 

If John has'TD?', how much does Ann have? 



131 



W3 Miscellaneous Word Problems 

Abstract: Solve word problems which are unusual for third graders in • 
one of several ways: requires three-stage solution, requires 
working backward from a given final state to an unknown initial 
state, requires more logical analysis than straight computation, 
involves proportional! ratios, involves extraneous data. 

Grade Level : 3 ^ 

Examples: At first, Sally had some marbles. 
Then, she lost 3 of -them. 
Then, she found 2 marbles. 
After that, she still had 8 marbles left. 
How many did she have at first? 



Sam has to move 10 boxes. 

He can carry 3 boxes each trip. 

How many trips will he need to make? 



•W4 Extraneous Information X 

/' 

Abstract: Solve word problems in which extraneous information is given. 

Once the relevant information is selected, the solutions are 
' . simple one-step problems involving small whole numbers. 

Grade Level : 4 * 

Examples: A belt costs 54. 

A shirt costs $5. 
A. hat costs 510. 

How much more does a hat cost than a belt? , 

Peter has JIG. 

He needs 4 pounds of candy. 

Candy is $2 per pounfl. 

He is buying candy for ^6 people. ^ 
How much will the candy cost altogether? 



W5 Fractional Sugar \ 

Abstract: Solve^word problems each of which start with cups of Sugar. 

The one-^step* solutions all require simple computions (+, x 
or •) with fractions or mixed numbers. 

ji 

Grade Level: 4 , > ' ^ 

Examples: Tina has *T cups. ( 
She buys 5*^ more cups. 

How much sugar will she have then? 



Kari has 4-jr cups. 
^ She gives away Kalf of it. 
'gl^Q , ' How many cups of sugar, will she have left? 5ij 





• 


• 

• 




W6 .Three Stage .Word Problems 


• 




Abstract:^ Solve word problems^ in which the solution requires three ope)ra- 
tions. The problem is stated, in 3 to 5 short sentences and the 
numbers given in the problems are relatively- small. 






Grade Level : 5» 6 


• 




Examples: Shirts cost $10 each and ties cost $5 each. 

Altogether Joe spent $35 for shirts and ties. 

He bought 2 shirts. 

How many ties, did he buy? 






Bin loads 6 boxes in 2 hours., ^ 
John loads 4 bojces in 2 hours. - ^ 
Together, how many boxes do they load in 6 hours? 

JfiIjU--Be?Tma1 Gas 


• 




« 

Abstract: Solve word problems each of which start with 6.5 gallons of 
gas. The one-step solutions all require simple computations 
(+» X, or -) with decimals. 


• 




Grade Level : 5 . * - 

V 


J 


< m 


Examples:; Peter has 6.5 gallons. 

Then he spills 1.2 gallons, 
f How much qas will he have left? 






Ron has 6.5 gallons. 
^ Next week he-will use ten times this much. 
How much qas will he use next. week? 


• 




. W8 Novel Word Problems 






Abstract: Solve word problems which are .novel for sixth graders in one^or 
two of the following ways: involves fractions or decimals, 
requires more-thanrthree-sta^e solution, answer choices are^^ 
approximate, requires s,olving for two unknowns, requires the 
use of data which is. common knowledge but not given in the 
problem. Response>f ormat isfe-mtritipls choice. 


• 




Grade' Level: 6 - ' 
- Examples: 

Ellen saw pepper plants on sale at 3 plants for AO^.^ 

She bought 12 plants. 

She usually bought 3 plants for SO^. 

How much did she save? , 

20i "W 48;^ JU60 J2.00 


• 
• 
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Oeorge's father gives him 2sf for every hour he spends in school. 
About how much would he have given George for the months of October? 

5.50 Jl.OO $3.00 $6.00 . $10.00 t 





Category A: Algebra 



Al Algebraic Symbols 

Abstract: Given \he numerical value of a letter (or letters) produce t^e 
numerical value of aa expression involving that letter (thdK* 
letters). In written instructions, two sample items are w(Med 
out and Implied multiplication (e.q. in 3bc or in d*) is 
plained. This scale follows A2 in"the test booklet. 

Grade Level: 6 * ' . * ' 

Examples: If g » 4 and h » 3 then 5gh « 

If p » 2 then p5 = 

A2 Solving Equations 

Abstract: Given simple equations in one unknown, solve for the unknown. 

, Three sample items are worked collectively, including one with 
a parenthesis. 

* 

Grade Level : 6 

Examples: (7 x h) + 1 » 15, so h » 

(n + 1) 3 », 6, SO n = 

A3 Summation Operajtor ' 

' Abstract: Given an open. sentence involving one or more summations of 
consecutive^.integers, select the answer that completes the 
sentence. A symbol for such summations (^T^) is introduced 
and explained (<S>-r<2)s 2+3^4 45^^>'\ and two items are worked 
collectively. 

Grade Level : . 6 
.Examples: 



■.1 b. n e. 100 i. ]n 
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A4 Transformations 



Abstract: Given two different trans forrot ions ("=1 which turns a design- 
clockwise by 90O and X which reverses the number of symbols 
at the top and bottom of a design), the scale consists of two 
different sections: requiring the application of either 1 or 
X to a design, requiring several applications of '^l-^.and/or X, 
to a design. Several sample items, are worked collectivelyc 
in each section. 

Level : 6 ^ * " 



Grade 



Examples: Section I 



Stirt with 



Section II: 



End up with 



do X ^^•^ ^ 



K 
O O 



do X 



twice 
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Category G: Geometry 



Gl Geometric Loci 

^ Abstract: Determine which picture is described by a given statement,' where 
several pictures are given, each of which has identically placed 
elements (an 'x/ an 'o' and a line) but a different set of 
dotSt determine which picture a given statement describes. First 
statement is r^ad by the tester. 

Grade Level: 4 
Examples: ^ 





In which picture arfe all the dots the same distance from the x? A B C D 
In which picture is each dot just as close to x as to o? A B E F 



"62 Geometric Co^ngruencies • ■ 

Abstract: Given a regular geometric shape dividethe shape into a certain 
number of congruent parts. The word "congruent"is not used. 
Three correct and three incorrect solutions to a sample pnoblem 
' are examined collectively. 

jQrade Level : 5 
Examples: 




0/ 
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9 



Category G: Geometry 



63 Geometric Categories 

Abstract: Given nine different geometric figures, identify a set of 2 to 7 
figures that are alike in some way, describe^-the distinguishing 
characteristic and label the figures accordingly. Go through 
this process as many times as possible. Two examples are worked 
collectively. 



Grade Level : 6 
Examples: 





( 





J 






Stcplt 1 An tht figurts with "A* KfXWg. *iq\xf\/9^ Qw^.lrA ' 

Swjplt 2 All tiff ff^urts idth.^B" VsxsiK hi)<?f)tdto tUg-f Qt/^ iMrJ. foty f ^. 
All tilt figurts with "C" . - 



All tli« f1gurtj>.w1th 'D' 
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etc. 



Category L: Aogic 

LI Logical Identification 

Abstract: Given a specific set'of individuals, a specific set of character- 
istics, the fact that each individual has a distinct combination 
of characteristics, and several facts about some of the charac- 
teristics of some of the individuals, identify the characteris- > 
tics of each individual. A smaller sample problenf is worked 
collectively. 

Grade Level : 6 



Example : 4f« 4 »tn: nn im u hu 

Tutt* Art 4 iMfvti. iMMf Mccar mfo^r tHt9r \im9r K«aty tut«Otr ^ttr 

TMtM tf« IM f«€U: iMCfi hf pltyl iK 4 «1 f f trffit 1t4fvC. 

nil »1t]r» Umtrt ^ 

U <cfiwt pity wimmn m4 m #»ti»'t »lty itectr« 

«i4t Ittfwt «M« ttcn toy pity (Cirtit ytvr tftwtn.) 

till: in4mr itccff twtMf mctr i tttiP tocttjr Mt^Mf Itaty 

T««: i««t«r itccvr Mttttr »tectr Immt ft^cMy tvtCMr i^ty 

U'. iMMr itcttr ifUmir ttectr 1«#Mr toCMy Mt<t*r f»tctty 

♦ 

L'2 Making Sentences False 

Abstract: Given a picture of a set of blocks and a true sentence about 
them, make the sentence false by changing the^blocks. In the 
y first two items, three suggested changes in* the -blocks are given 

and the student need only mark which ones would falsify the 
sentence. In the last three items , the student must write a 
change in the blocl^s. An item of the first type is worked 
collectively. 

Grade Level : ^ 6 , 

■ ^"""'"^ OADAD 

JOE'S ILOCW — ^--->J--» 




□ □ ^ 

"There are triangles above the line and squares below the line.' 

a. Take away the triangles. 

b* Take away the squares below the line. 

c. Add squares above the line. 

•Triangles go above the' line or circles go below^the line." 
(You write what Joe could do to make the sentence false.) 



Category 0: Organization of Data 



01 Graphing Weight ^ 

Abstract: Given a graph in which weight (axis labelled at 10 pound 

increments for each 5 graph units) is plotted against age (axis' 
labelled at 2 year increments for each 2 graph units), determine 
age per given weights and vice versa. One sample item is worked 
collectively. 

Grade Level: 5 

Examples: How much did Bill weigh at 4 1/2 years, of age? 

How old was Bill when he reached 90 pounds? ■ 



02 Interpolating from a Table 



Abstract: 



Given a t^ble of prices for pipe of 4 different widths and 4 
different lengths, interpolate or extrapolate to obtain the 
price on a pipe of given dimensions: at-^least one of which is 
not shown in the table. Two sample items are worked collectively. 



Grade Level 
Examples: 



> CDst of Pipt 



I 



Ltngtt) 
O' 600* 



KOOO' 



Width 



4- 

ft- 
12" 
1$' 



$50 
S70 
$90 
SI 10 



150 
$210^ 
S270 
$330 



$300 
S420 
$540 
$660 



$500 
S^700 
$900 
$1100 



HOW MUCH DOES IT COST TO BUY PIPE WHICH IS; 

6" X 100' = $ 



20" X 1000' 

4 



1 



it ' 
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Category P: Probability 



PI Choosing the Best Box 



AbstracJ: .Given three boxes containing different combinations of 1, 2 and 
50-cent "balls", determine from which box it would be best to 
make a blind draw. 



Grade Level : 5,6 
'Examples: 



WHICH BOX WOULD YOU CHOOSE? 



® 
(g) 
® 
© ® 





® 




@ 


© 


® 


® 


® 



® 
® ® 
© @ 
© @ 



WHICH BOX WOULD YOU CHOOSE? 



© © 










'® 


©.© 


® 








© ® 


©©. 


® 




® 




© @ 


© © 


® 




© @ 




© @ 



P2 Dependent Outcomes 

Abstract: Given two (or three) spinners and an amount (10) to be achieved 
or exceeded to. win, select (from five standard choices) how 
often a player would win. Collectively it Is shown how a player 
could win or could lose with a specific set of spinners* 



Grade- Level ; 6 
\ Examples: (^l^) (q\^ 



two forms, approximately 3.5 minutes. 



Mvtr 



Ittt thtn 
t1a« 



h«1f 
. ttit 
tlw 



■on th«n 

tiM 



•1«r«ys 



Utt th«n 
tim 



t1«t 



■Oft than 
hilf Vm 
tim 



always 



Appendix B 



Graphs o'f Class Means by Grade ^ and by Category 



I 

Note to reader: It was no^ possible to prepare 
these prior to the publication of this draft 
report. 



ERIC 



Appendix C 



List of SchoQl 
Participating in 



Ann Arbor, Mi^.chigan 
Baltimore County, Maryland 
Bedforfl, Michigan 
Bronx, tiew York 
Clarksville, Tennessee 
Detroit, Michigan t 
District, of Columbia 
Ferguson-Florissant, Missouri 
Gillette, Wyoming 
Glendale, Wisconsin 
Globe, Arizona 
"Grinnell, Iowa 
Guilder land, New York 
Harrisonvil le, Missouri 
Hartsdale, New York 
Hawaii Department of Educatiori^ 
Janesville*, Wisconsin 



Districts 
MANS Testing 

Ladue, Missouri 

Louisville, Kentucky 

Madison, .Wisconsin 

Maplewood-Richmond Heights', Missouri 

Marquette, Michigan 

Mississippi State, Mississippi ^ 

New HartfoVd, New York 

New Orleans, Louisiana 

Normandy, Missouri 

Philadelphia, Pennsylvania 

Polk County, Georgia 

Portland, Maine 

San, Felipe, New Mexico 

St. Louis, Missouri . 

St. Louis ParocbHal, Missouri 

University City; Missouri 
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Appendix D 

Evaluation Panel 

- - • 

Ernest House (Chairman), University of Illinois. 
Robert. Dih/orth, California Ins-titute of Technology 
Leonard. Cahen, Arizona State University • 
Peter Hi Hon /state University of Ne,w Yorlc, Binghamton 
Stanley Smith, 6a,Uimora County Schoo'ls ' 




